Deriving lessons learned from monitoring adaptation activities in projects under the EU mission on adaptation

Actions to strengthen climate resilience are gaining more traction. In order to ensure effective adaptation, it is important to monitor the outcomes and impacts of these actions. However, there are numerous challenges and a multitude of approaches when it comes to monitoring adaptation to climate change. This paper addresses challenges in setting up mechanisms for monitoring climate resilience and adaptation projects. Drawing from three EU Horizon 2020 projects under the EU Mission on Adaptation to Climate Change, it synthesizes challenges and insights to support future initiatives in their monitoring endeavors for other projects to learn from. Findings, acquired through workshops with experts who shared learnings and challenges, highlight four key themes: the challenge of tailoring global frameworks to local needs, data availability and evaluation of data, interdisciplinary collaboration in monitoring, and stakeholder engagement for monitoring endeavors.


Introduction
Monitoring climate resilience is gaining importance on the global and on the European agenda especially as its significance is highlighted in the EU Strategy on Adaptation to Climate Change.The strategy's objective is to engage more regions in adaptation initiatives and to facilitate the transfer of locally developed solutions to a wider regional and national context (European Commission, Directorate-General for Climate Action, 2021b).The EU Mission on Adaptation to Climate Change builds upon this strategy and was adopted to support regions across Europe in achieving climate resilience by 2030.Under the umbrella of the mission, there are a number of applied research projects that assist regions in implementing adaptation activities and monitoring progress towards enhanced climate resilience (European Commission, Directorate-General for Climate Action, 2021a).
Monitoring and subsequent evaluation are the fundamental requirements for understanding which adaptation actions and policies prove successful and for comprehending how climate resilience changes over time.Additionally, sound monitoring can contribute to the effectiveness of adaptation measures, can strengthen accountability and allows tracking progress as well as outcomes and impacts.Furthermore, monitoring and evaluation at the regional level allows drawing lessons learned from implementing transformative actions for risk reduction and resilience enhancement.Given that various sectors, such as transport, water and healthcare, often fall under regional government jurisdiction, the importance of monitoring at the regional level cannot be overstated (Setzer et al., 2020).This is why under the Mission on Adaptation, the Regional Adaptation Support Tool (RAST) was developed, which emphasizes that monitoring should gauge progress with respect to: reducing climate impacts, reducing risks and vulnerabilities and increasing adaptive capacity, meeting adaptation priorities and addressing barriers to adaptation.However, designing a monitoring (and evaluation) framework that comprehensively measures development towards climate resilience can be challenging for various reasons and there is not one approach that fits all contexts to monitor and evaluate adaptation actions taken (ClimateADAPT, 2023;Pringle, 2011).The differing impacts on various fields, such as infrastructure, economy, communities, institutions, ecosystems etc., outline the different dimensions that a monitoring approach has to cover, which is only one reason why multiple approaches are needed.
Therefore, this document presents a collection of challenges and lessons learned grouped into four themes, namely (i) tailoring global frameworks to regional/ local needs, (ii) data availability, applicability, and evaluation, (iii) interdisciplinarity in climate adaptation and (iv) stakeholder engagement.These themes were derived during the development of a monitoring (and evaluation) scheme for resilience and adaptation measures by three early EU Mission projects (ARSINOE, IMPETUS, TransformAR), referred to as projects that have started implementing resilience measures in vulnerable regions.As the projects are still ongoing until 2025, the following are initial observations focusing on the preparatory and early implementation phase of monitoring activities.The project REGILIENCE initiated the gathering of lessons learned from the three projects via joint workshops with experts from the projects to support future mission projects in their monitoring endeavors.The document thus functions as an information source for any project and region conducting monitoring (and evaluation) activities.It provides inspiration and preemptive considerations for future Mission projects and is intended to help circumvent previously identified difficulties to avoid common pitfalls.Eventually, this can contribute to a more efficient and streamlined use of project resources.

About the three projects
In the following, the three projects of which the learnings, experiences and challenges stem from, are described together with their monitoring approach.
ARSINOE project and its approach to monitoring climate resilience ARSINOE aims to leverage innovative, cross-sectoral climate change adaptation solutions, as well as leveraging regional databases and climate and impact simulations.The project involves collaborations among various stakeholders across the quintuple innovation helix (Carayannis et al., 2012), including academics, authorities, municipal companies, agriculture, forestry, water and environmental protection groups, technology providers, urban planners, and citizens.Through a living lab approach, the initiative fosters a shared understanding of the impact of climate change.ARSINOE develops data analysis tools and models to facilitate the design of adaptation strategies and measures.Hence, it contributes to resource management, energy, water and food security, and preserving ecosystem functions and services.

Monitoring ARSINOE's monitoring work is based on the Sendai
Framework and the Sustainable Development Goals (SDGs) as overarching frameworks.The SustainGraph is a knowledge graph that monitors resilience on a broader level -to the extent where resilience relates to sustainable development -and monitors progress towards achieving SDG-targets.This tool acts as a unified knowledge source, leveraging graph databases and machine learning techniques for data population, knowledge production, and analysis.It maximizes the use of available data, ensuring openness and interoperability with existing databases and Application Programming Interfaces.The SustainGraph facilitates participatory modeling and analysis processes for socio-environmental and socio-ecological systems (Fotopoulou et al., 2022).It aligns with the principles of a Systems Innovation Approach (Schuurman et al., 2023).Additionally, the project intends to develop a Multi-System Dynamic Modelling Framework for Resilience Assessment, by the end of the project, that will integrate the various modelling subsystems that have been developed in various academic disciplines and use discipline-specific methods, tools and techniques, enabling transdisciplinary modelling of both natural and man-based systems.

Demo-site level monitoring
ARSINOE is also employing a bottom-up approach for demo site-specific monitoring.This is achieved by establishing "living labs".Stakeholders, identified through the quadruple helix (based on interest and influence of stakeholders; only highest ranked stakeholders are involved), collaboratively define a problem statement and a vision that outlines solutions to address the identified issues.As a next step, backcasting creates a future narrative that gradually moves backward to identify innovative strategies and actions for the region.This process establishes monitoring needs by considering resilience goals for the future.Living labs facilitate the development of monitoring strategies specific to case study regions.Indicators are developed with the involved stakeholders, ensuring a comprehensive and participatory approach to resilience planning and assessment.Variable case specific monitoring technical solutions are applied, such as ground sensors, satellite data, drone missions, and citizen science applications, depending on the nature of the monitored variable, the demanded temporal and spatial scales, and technical requirements.
TransformAr project and its approach to monitoring climate resilience TransformAr's primary goal is to demonstrate how co-innovation processes can drive transformational adaptation towards climate resilience in vulnerable regions and communities across Europe.This involves six demonstrator regions to develop, test, and scale products and services that catalyze significant adaptation efforts.

Monitoring
TransformAr employs a top-down approach by modeling climate resilience at the NUTS2 level, considering climate risks and adaptive capacity.This will make climate risk information available for a wider audience.By integrating socio-economic data in the process, transformative adaptation at the regional level can be assessed.

Demo-site level monitoring & evaluation
Simultaneously, a bottom-up approach is applied that involves developing indicators at the local level to enhance the granularity of the monitoring process.This is done through stakeholder engagement.Additionally, by applying choice experiments and developing a rapid cost-benefit analysis tool, further data is generated to evaluate climate adaptation measures.Furthermore, one demonstrator region develops a resilience index which is focused on aquaculture and encompasses the entire value chain.
IMPETUS project and its approach to monitoring climate resilience IMPETUS employs Resilience Knowledge Boosters (RKBs) to create scalable and multi-level open knowledge spaces offering opportunities for experts, key communities, and quintuple helix stakeholders to share experiences and knowledge for implementing dynamic pathways and packages for climate adaptation.

Monitoring framework
IMPETUS has an overarching approach to monitoring, that started with the development of a flexible indicator-based monitoring framework built upon literature and stakeholder input.The IMPETUS "Metrics for climate change vulnerability, resilience and adaptation" are suitable for all the European biogeographical regions by accommodating their diverse characteristics.It encompasses two core frameworks addressing climate vulnerability and climate adaptation potentially applicable to all demonstration sites of the project and to most situations.They are organized into different categories and subcategories and are searchable by sector and impact.An additional 43 indicators, not less relevant than the 69 core indicators but with more specific application potential, complement the framework.Serving as a structured tool for climate-sensitive decision-making and as an indicator repository, this framework provides a meaningful starting point that can and should be adapted based on new learnings and contextspecificities at local and regional level.

Monitoring on demo-site level
Based on the catalogue of metrics, the local-level demonstration sites will select stakeholder-customizable indicators and metrics that fit their context and eventually add additional sitespecific indicators.Selected indicators will feed the IMPETUS Regional Climate Resilience footprint tool, that will allow stakeholders to assess regional resilience and its evolution over time, capturing the effect of adaptation interventions.Within IMPETUS, Resilience Knowledge Boosters RKBs indicators are seen as tools to predict, verify and continuously reassess the effect of alternative adaptation options and adaptation pathways in different demonstration sites, supporting decision makers in the adaptation process.Stakeholder engagement is a key action implemented by the project to enhance local knowledge and support the identification of local-scale indicators.Stakeholders of the seven IMPETUS demonstration sites started to tailoring the indicator framework to their needs, considering a very large variety of climate change risks: flooding, water scarcity, marine storms, fires, biodiversity loss, health diseases, temperature increase, avalanche increase, and extreme storms.Indicators are meant to be used in assessing alternative adaptation pathways to enhance the resilience of key systems in demonstration sites, allowing for flexible and dynamic decision-making.

Global frameworks
In the process of developing their individual monitoring approach, all three IA projects started off by screening existing and acknowledged global frameworks, such as the Paris Agreement, the Sendai Framework for Disaster Risk Reduction, the Agenda 2030 and its SDGs, the Lancet Countdown, the One Health approach and the Water-Energy-Food Nexus approach.
The Paris Agreement defines a global goal on adaptation (Art.7) that features three core components: enhancing adaptive capacity, strengthening resilience, and reducing vulnerability to climate change.These are positioned in the overall context of limiting global temperature rise as close as possible to 1.5 degrees Celsius compared to pre-industrial levels.Discussions on potential approaches to assessing the global goal on adaptation have started with a technical paper stating the need for adaptation monitoring and evaluation (UNFCCC Adaptation Committee, 2022).During COP28 these endeavors proceeded and the UAE Framework for Global Climate Resilience was adopted which sets the target for all parties to establish a monitoring and evaluation system for national adaptation efforts by 2030, ensuring sufficient institutional capacity for implementation (UNFCCC, 2023).
Country-specific approaches combined with subnational-level approaches are estimated as potentially useful in tracking progress towards the Global Goal on adaptation.Further, the Sendai Framework on Disaster Risk Reduction (2015)(2016)(2017)(2018)(2019)(2020)(2021)(2022)(2023)(2024)(2025)(2026)(2027)(2028)(2029)(2030) given its clear connection between climate change adaptation and disaster risk reduction.The link between disasters and adaptation originates in the increased risk of extreme weather events, including floods, droughts, and wildfires, as also pointed out by the EU Strategy on Adaptation to Climate Change (European Commission, Directorate-General for Climate Action, 2021b; UNDRR 2015).Furthermore, the Agenda 2030 and its SDGs include indicators that touch on synergies between climate resilience and adaptation, with particular reference to the SDG 13 (take urgent action to combat climate change and its impacts).Indicators from other SDGs are also relevant such as SDG 2 (end hunger, achieve food security and improved nutrition and promote sustainable agriculture), SDG 6 (ensure availability and sustainable management of water and sanitation for all), SDG 11 (make cities and human settlements inclusive, safe, resilient and sustainable) and SDG 15 (protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and half and reverse land degradation and halt biodiversity loss).These highlight the cross-sectoral nature of climate adaptation and resilience (United Nations, 2015).Moreover, the Lancet Countdowntracking progress on health and climate change -is another global key reference for monitoring adaptation through an indicator-based approach (The Lancet Countdown, 2023).It puts a focus on the effects of climate change on health.
The One Health approach addresses the complex interactions between climate change, human health, animal health, and the environment and thus provides a source of knowledge for informing climate resilience monitoring.Furthermore, the Water-Energy-Food Nexus approach was acknowledged, recognizing the interconnectedness of water, energy, and food systems for a more holistic assessment.Highlighting that the interdependence of these resource systems is valuable in monitoring climate resilience, as changes in one sector can have significant cascading impacts on others, both positive and negative.Given the exacerbation of pressure on these sectors' resources by climate change, incorporating this approach is thus valuable when developing climate resilience monitoring schemes to ensure the consideration of the interconnectedness of sectors when monitoring climate resilience (Food and Agriculture Organization of the United Nations n.d.).
Indicator-based monitoring approaches are useful to policy-and decision-makers at the regional governance scale, and they facilitate knowledge-based stakeholder engagement.However, existing global indicator frameworks pose several limitations and challenges that have been widely discussed in several academic and technical papers (e.g.(Bours et al., 2014;Hammill et al., 2014;Leiter & Olhoff, 2019;Stadelmann et al., 2015;Sanchez Martinez et al., 2018;UNFCCC Adaptation Committee, 2022;Vallejo, 2017).An analysis of these limitations and challenges can be found in the "Metrics for climate change vulnerability, resilience and adaptation" report by IMPETUS (Koop et al., 2022).The main limitations can be traced back to the following aspects: -Climate change is global but adaptation takes place on a local level: Indicators and metrics should be site-and context-specific which cannot be reflected in universal frameworks like the aforementioned.
-Interconnectedness of adaptation and vulnerability: Data that is used in global metrics does not reflect local/regional adaptation initiatives and their impacts -Adaptation monitoring lacks clear targets: Adaptation lacks common measurable targets due to its ongoing and dynamic nature.Unlike mitigation, there is no clear endpoint, making monitoring complex.
-Lack of agreed baseline to assess changes: The absence of a well-defined baseline hinders assessing the impact of interventions, as the overall context is dynamic, requiring more than a simple 'before' and 'after' comparison.
-Complexity in measuring adaptation success: Measuring the success of adaptation is intricate, involving the quantification of "avoided impacts" and dealing with time lags (between intervention and measurable impacts), attribution challenges, and the potential for long-term changes.
-Maladaptation not sufficiently reflected in indicators: Indicators may not adequately signal maladaptation, as they measure adaptation progress but often fail to assess the overall quality, environmental sustainability, and potential negative side-effects.
-Limited explanatory power of indicators: Indicators primarily reflect progress or change but often lack the depth to explain how, why, and what improvements could be made.Understanding the overall adaptation process behind the numerical value expressed by the indicator is crucial.
-Resource-and data-intense nature of monitoring: Monitoring adaptation demands significant resources, including suitable data and technical capacity.Barriers include the lack of long-time series for certain variables, decentralized data, and variations in calculation methods that hinder the uptake of indicators from international frameworks.

Learnings of the projects
Expanding upon the challenges outlined previously, which were identified by the IMPETUS project through literature screening, this chapter delves into the practical challenges and lessons learned derived by the three projects during the development of their respective monitoring schemes.These are preliminary learnings as the three projects are ongoing until 2025.
From global indicator sets to demo-site specific indicator sets and metrics All three IA projects are following the approach of grounding their monitoring work on existing frameworks and approaches (the ones named above) and tailoring them to demo-site and project specific needs.-Allocate resources (time and financial): Monitoring demands substantial resources within a project, encompassing both time and financial investments.This concerns the phases of the project's inception and the design of the monitoring approach and ideally extends beyond its implementation.It is crucial to explicitly acknowledge and reflect this reality in project planning document as early on as possible to ensure adequate allocation of resources and realistic expectations regarding monitoring efforts.

Data for monitoring
As mentioned before, measuring climate resilience can be highly complex.Especially when the aim is to tailor monitoring frameworks to regional and local contexts.It requires substantial amounts of data which covers different temporal and spatial scales to allow long-term monitoring and evaluation: -Screen data availability at the start: Finding easily accessible and suitable datasets for the calculation of certain indicators at regional and local level can be challenging.It is crucial to assess data availability at the regional level early on (ideally right at project start) to avoid developing an indicator framework that might not be put into practice due to data constraints.Specifically, socio-economic data, e.g.data on knowledge and education/ financial resources etc. is often not available in a high spatial resolution and is recorded with a lower frequency compared to biophysical attributes (e.g.NUTS3).There is thus a clear need to strengthen data collection endeavors at the beginning of a project.Baseline data plays a critical role in attributing changes to adaptation measures, but its adequacy and accessibility are key.
- This entails promoting a holistic understanding of the complexity of resilience and adaptation as well as breaking down disciplinary silos.Recognizing that each hazard entails a complex and intricately connected system, the explicit interlinkages are yet to be fully understood and for that, interdisciplinarity is a tremendous advantage.

Stakeholder engagement
The human dimension of monitoring climate resilience is crucial.Involving stakeholders is essential for selecting the most suitable indicators and identifying existing databases as the specific context and local specificities are best known by stakeholders.
-Strategically engage stakeholders: In the projects, different formats of stakeholder engagement were employed, some of which were also used for developing monitoring indicators.A successful approach strategically includes stakeholders identified as high ranking in both influence and interest.The Quadruple and Quintuple Helix approach used in the projects recognize the importance of a broader participation in decision-making processes and innovation activities (Braun et al., 2021;Carayannis et al., 2012).Not only representatives from government, industry and academia (triple helix, the knowledge of economy) but also the civil society (quadruple helix, the knowledge of society) and the natural environment (quintuple helix, the knowledge of the environment) have been considered in the stakeholder engagement activities.
-Identify blind spots: Stakeholder engagement proves valuable in discovering previously overlooked aspects of climate resilience.Subsystems that were not initially recognized or considered important are revealed through stakeholders' hints, especially through identifying interlinkages of vulnerabilities, compound hazards and cascading risks.Incorporating stakeholder input adds nuance and depth to the identification of monitoring priorities, contributing to the development of a comprehensive list of indicators.Within the projects, different engagement processes were utilized.Some opted for living labs, where stakeholders collaborated to shape a shared vision for a climate-adapted future.
In these settings, stakeholders pinpointed critical subsystems crucial for building resilience in e.g. a demosite regarding a specific hazard -thereby helping to determine the most relevant indicators.This approach ensures that indicators align with the practical needs and priorities of those directly affected.Others used social innovation and participatory methods for exploiting the knowledge of different disciplines and backgrounds to develop indicators for monitoring.
The result is an enhancement in the robustness and relevance of the entire monitoring and adaptation process.
-Generate qualitative data: Moreover, stakeholders play a key role in providing qualitative data and narratives, contributing to a holistic understanding of adaptation efforts that goes beyond quantitative data, e.g.grounded in a theory of change.The primary objective was to identify positive impacts in terms of adaptation for each demo-site and articulate the essential adaptation pathways and indicators for measuring these impacts.Engagement formats such as workshops, small thematic focus groups addressing technical and policy aspects, and bilateral meetings were used.This flexible and adaptable approach allows for active co-creation with stakeholders, facilitating the identification of tailored indicators aligned with their unique adaptation pathways.Focus group discussions were instrumental in uncovering diverse pathways and narratives.

Conclusion and outlook
The approaches of the three projects on monitoring climate resilience have unveiled challenges and insights that other projects can learn from.Monitoring climate adaptation measures poses a unique challenge due to the interdisciplinary nature of the field, requiring effective integration of diverse perspectives, methodologies, and data sources, with key considerations including the clear definition of terms, the integration of different disciplines for shared understanding among all project members and stakeholders.Basing the development of indicators on existing global frameworks is advised, together with the contextualization of indicators to local needs.Accessible and available (baseline) data forms a fundamental element of this preparation.Equally critical is the implementation of a well-structured stakeholder engagement process from the project's inception, ensuring diverse perspectives and insights are considered.Additionally, proactive measures should be in place to guarantee the continuity of monitoring (and evaluation) efforts even after the project concludes.
It is recommended that the insights gained here serve as a catalyst for a wider dialogue within the Mission Implementation Forum, fostering a community of practice and extending beyond individual projects to engage the broader regional community in establishing effective monitoring and evaluation practices.
Braun  2020).The authors acknowledge many of these versatile purposes of M&E for effective adaptation.However, the discussion misses some aspects of the just nature of adaptation interventions, a minor omission that can be easily addressed.
2.The authors also recognize the challenges of monitoring adaptation, such as the lack of a standardised framework and the involvement of multiple sectors.However, the discussion lacks consideration of the different timeframes, often long-term nature of adaptation projects, as also the attribution challenges inherent to these projects.Incorporating these elements will provide a comprehensive and complete overview of the discussion on monitoring and evaluation in adaptation.
3.The methodology and the thematic analysis in the introduction are somewhat indistinct.While these elements are not always easy to separate, a bit more attempt to distinguish between the manner in which the 4 analytical themes were identified and what the 4 themes claim, would provide more bite and clarity to the current paper.Additionally, the introduction could be structured slightly differently to highlight what makes the 3 projects particularly interesting to study MEL for adaptation.
4.Currently, the paper focuses heavily on methodology, with less attention given to identifying an interesting puzzle/ research question from the three projects.The authors may like to frame the four thematic explorations from the expert group workshop as specific crosscutting issues within monitoring adaptation that genuinely demand explanation, rather than presenting them as general phenomena requiring sophisticated description.
Reframing the introduction in this way will engage the reader's curiosity by presenting issues that need to be explained rather than merely described.
5.It is difficult to comment on the 3 projects as these are large, complex, involve creative collaborations, and are currently ongoing.However, more empirical detail about the monitoring tools, techniques, and frameworks used in the projects is needed and would add the necessary specificity to the analysis.For instance, in discussing the ARSINOE project, the reviewer is interested in the indicators used for resilience planning and assessment.Additionally, it is unclear whether the authors suggest that monitoring and assessing resilience planning is the same as monitoring and assessing climate resilience.If not, the exact focus of the monitoring remains unclear from the text.Similarly, in the description of the monitoring approach in TransformAr project, the nature of the new data generated is not clearly explained.The discussion on the IMPETUS project is interesting but does not tell us what is novel about the approach adopted.More detailed information would help clarify these points.
6.A brief explanation, perhaps in the footnotes or endnotes, of terms such as Systems Innovation Approach, and Resilience Knowledge Boosters (RKB) would be helpful.
7.The section on global frameworks is slightly misleading because it includes challenges of monitoring adaptation that extend beyond global frameworks.A subheading or a separate heading for better sign-posting would improve clarity.
8.It would be helpful to know whether and to what extent the monitoring approaches of the 3 EU projects have addressed some of the challenges listed by the authors.Currently, the discussion on the monitoring approaches adopted by the three EU projects is separate from the overall context of the constraints in monitoring adaptation.Adding a paragraph linking the two would be beneficial.Alternatively, a simple table format could be used to show whether and to what extent the limitations listed by the authors have been addressed in the 3 EU projects.This will also nicely connect to the final section on learnings, where proposed solutions and policy advice emerging from the 3 EU projects are presented.
Is the rationale for the Open Letter provided in sufficient detail?(Please consider whether existing challenges in the field are outlined clearly and whether the purpose of the letter is explained) Yes

Does the article adequately reference differing views and opinions?
Yes Are all factual statements correct, and are statements and arguments made adequately supported by citations?Yes Is the Open Letter written in accessible language?(Please consider whether all subjectspecific terms, concepts and abbreviations are explained) Yes Where applicable, are recommendations and next steps explained clearly for others to follow?(Please consider whether others in the research community would be able to implement guidelines or recommendations and/or constructively engage in the debate) Yes Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Climate change adaptation monitoring and evaluation approaches.
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.title) is needed to focus on just one of these aims, I assume it will be in the challenges of monitoring.Reflection on the others could be made in the conclusions section perhaps?
The abstract is very brief and does not really communicate the aim or content of the paper.I would use a more formal type of abstract, i.At present the abstract suggests that the methods involved workshops with experts -after reading the paper this does not appear to be the case, instead, the paper appears to be a review of 3 projects.If workshops were used to gather data from those running the projects this needs to be explained somewhere.If this is a literature review of the outputs from the projects -that should be explained.Greater clarity in the abstract would help.
The paper is broadly drawing on lessons from projects which are in progress.I understand the desire to produce fast-track outputs from research, but I would ask is it too early to be evaluating projects that have not yet finished?The four-year long projects started October 2021-March 2022.At this point (June 2024) they will be about 2.5 years into their research which usually would not enable much detailed analysis of progress or findings.Given the typical timing of most large consortium research projects (i.e.y1 getting started/gathering data, y2, more data collection and preliminary analysis, y3, detailed analysis and preliminary findings published; y4+, analysis written up and published), I am unclear how key findings on monitoring progress on adaptation, or progress towards developing climate resilience can be gleaned from these projects at this midway stage.
In terms of readability of the paper it would have been good to add in some footnotes or endnotes showing the timing of the three projects i.e. 4 years starting in xxx date.Difficult jargon could be better explained at first use e.g.quintuple helix stakeholders.
It is difficult to critique the sections on the 3 EU projects, as these are vast, early stage, large consortium projects.There is some massaging of evidence to fit to the desired focus of this paper.For example, the ARSINOE project is measuring progress towards the SDG progress, there is no evidence that progress towards the SDGs equates to progress towards climate resilience ("The SustainGraph is a knowledge graph that monitors resilience on a broader level -to the extent where resilience relates to sustainable development -and monitors progress towards achieving SDG-targets").I am not sure if ARSINOE has or will undertake work to assess whether SDGs are accurate indicators of climate resilience?If it has, this text needs to be added.If not, the text aligning progress on SDGs with progress on climate resilience needs to be changed.
The IMPETUS project looks like it will develop interesting information on adaptation.It proposes to develop metrics of climate vulnerability and adaptation.The vulnerability metrics look very similar to the work of Brooks et al, 2005 ("The determinants of vulnerability and adaptive capacity at the national level and the implications for adaptation").The adaptation metrics follow the commonly used approach of monitoring government delivery of institutional adaptation initiatives.I am unclear how these are taking forward our understanding of monitoring adaptation (beyond[1],[2] ). Documenting the state of adaptation for the global stocktake of the Paris Agreement.Wiley Interdisciplinary Reviews: Climate Change, 9,5, e545.).Some additional text explaining clearly to reader the additional contribution made by this work would be useful.
The section on global frameworks offers some reflection on the limitations of monitoring adaptation, however most of the limitations listed could be gleaned from extant academic sources on the challenges of monitoring adaptation (see[1]etc.. above).

Diana Reckien
University of Twente, Enschede, The Netherlands Dear Authors, thank you very much for this insightful paper.This is a timely topic and studies on the lessons learned after implementing climate responses are crucial.
I have a few suggestions how to improve the paper: -in the abstract, you mention that the workshop yielded insights into "four themes."This is raising a question mark.Could you please be more specific: do these four themes relate to challenges or "insights" in a positive way?
-in the third paragraph of the article, you refer to the RAST system and the difficulties of applying a generic framework to different regional contexts.The reference that you cite, "Prinlge, 2011," is rather old.In that regard, please include more recent references, e.g.look at the latest IPCC Report AR6 WGII Chapter 17, there are particular sections on monitoring and evaluation as well as a Box that discusses the difficulties of tracking progress.In these parts of the IPCC AR6 WGII you find a lot of more recent references, including the ones by Leiter et al.
-Before you can move to the fourth paragraph in the introduction, you could provide more insights into how you arrived at these four topics.It sounds like these were coming out of your workshops; hence, I would report those as results.But in this case, the wording here in the introduction needs to be changed.In the introduction, it sounds like you derived these four themes from a literature review.For these four themes to work as a synthesis of the literature, you need to provide more background/ literature review.This is currently missing.So, either add that or only report these as an outcome of the workshops in the results'section.
-After the introduction, you go into presenting the projects and then into what I would call "results," i.e., by comparing the projects and their approaches, frameworks, etc., of monitoring.In the abstract, however, you mention that workshops were held.I suggest adding a small section on "Methodology" to get a better overview of how you assessed these projects, why you selected these three, what kind of indicators you choose to compare these projects and why, who joined these workshops, how many people were there, what was discussed at these workshops, etc.
-I regard the section on "Global Frameworks" as a bit superficial, i.e., it does not go deep enough to explain why these frameworks were chosen and which indicators were useful or not useful.On the other hand, this section is not needed for the article topic.All the information provided in this section can be looked up elsewhere.I would shorten this section substantially, i.e. it is enough just to state the projects used these different frameworks--the current additional text is not needed or would need to be more detailed to be helpful.
-The issue "Interconnectedness of adaptation and vulnerability: Data that are used in global metrics does not reflect local/regional adaptation initiatives and their impacts" has issues: the information before the ":" and after it do not correspond to each other.Please provide an alternative description after the ":" -in the section "Global Frameworks", para #3, sentence #2 there is a verb missing.Rephrase.
-in the section "Learnings of the project" you repeat a lot of the information that is given in earlier sections, e.g. in the global frameworks section.I would shorten this section to avoid redundancies.E.g., the bolded keywords in the beginning of each item in the list (e.g."Specify indicators"; "prioritize indicators") is a good take-away/ a learning from the project, but what you provide as description and explanation after that (not bold) is in parts a repetition with earlier text.
-the text after "evaluation of data" is not related to learning.It describes the challenges but not the solution.In relation to my previous comment, it would be helpful if you keep the focus on the learning and not on the problem.E.g. it would be helpful to know whether you derived those "learnings" from reviewing data/ workshop notes or whether these were mentioned by the workshop participants themselves.How did you get to these learnings is a general question that I have.This question will need to be answered to assess the validity of your findings.
-being a bit more critical and detailed to the learnings as presented in the section "stakeholder engagement" is particularly needed, as this is usual a critical point.I.e., how do you strategically engage stakeholders?Was that done successfully in any of these projects?Who said that?
-general, I am asking myself whether all these learning come out of all three projects assessed or only two of them?How do you derive your findings/ results?-in the section on "identifying blind spots" you mention "some" which raises the question who these 'some' are?Please can you be more specific.
-in general, in this section on stakeholders, please bring examples, i.e. who exactly said what? E.g. "different engagement processes were utilized".this is a very generic sentence and needs further specification to be useful as a result/ learning."some" and "others" sounds like you had a lot of feedback.But you are only reporting from 3 projects.So, who are "some" and "others"?
-The section on "generate qualitative data" is insufficiently detailed.For example, what kind of qualitative data can be used?The description provided re "engagement formats" does not prove why qualitative data is important; these formats could also be used to collect quantitative data.When saying, "focus groups were instrumental .The Letter's intention to inform through an account of challenges encountered, and to support learning that benefits monitoring practices, appears to be well-fulfilled.Notable within this is the centrality given to multi-stakeholder engagement, both top-down and bottom-up, to develop location-specific indicators through which to monitor adaptation projects.This is true of each project presented.
A key argument conveyed within the Letter is the importance of qualitative data.This, it is suggested, can be particularly useful when quantitative (baseline) data are lacking through providing insights into the local or regional socio-cultural and political contexts around adaptation projects ('Data for monitoring' section).This also seems appropriate and useful.
There remain a few additional observations, and some minor suggestions, to make.
Firstly, some of the terminology could be clarified for greater accessibility for readers who may be uncertain of what certain phrases mean.This is perhaps the case with mention of the quadruple and quintuple helix when introducing the ARSINOE project.These terms are clarified within the penultimate 'Stakeholder engagement' section, but could be helpfully defined earlier in relation to ARSINOE.Similar stands for the Systems Innovation Approach principles that could be briefly outlined for added clarity (also in the ARSINOE section).The abbreviation 'IA' could be clarified on first usage, too (Global frameworks section).Such definitional work has been well-captured within other parts of the Letter, including around IMPETUS' 'Resilience Knowledge Boosters' and throughout the Global frameworks section.
The latter presents a very clear and succinct account of international agreements and frameworks, such as the Paris Agreement, while being up-to-date through reference to COP28 outcomes relevant to adaptation.Could there be more to say here about the EU Strategy on Adaptation to Climate Change?Currently the Letter uses this to underline disaster risk and adaptation linkages, but perhaps consideration could be extended by noting some of its core points within the wider EU Green Deal context.These include, for instance, the significance of monitoring and existing data availability limitations that speak well to the Letter's reflections.
The Strategy also discussed improving monitoring through developing a 'harmonised framework of standards and indicators' (EC (2021) Strategy on Adaptation to Climate Change, section 8) -this is something the Letter and the three projects can directly engage with in a meaningful, practical way through their learning points and suggestions.This is perhaps most clear in the Letter's treatment of global indicators (Learnings of the project section) in which they are complemented for their comparability while stressing a need for clear harmonisation methodologies and localspecific indicators that could be more insightful; key and useful suggestions.
Overall, the account of these frameworks' limitations is also a positive and this Letter does well to summarise some key adaptation challenges in an accessible way, inclusive of maladaptation that could be important for future learning and monitoring activities.By extension, this discussion helps to underline the possible impact of the projects and related reflections.
It was interesting to read about the living labs and their use would seem appropriate where the purpose is to develop locally-relevant and/or shared monitoring indicators with a range of stakeholders.This is a collaborative approach that could certainly be of use for future projects and initiatives, including those aiming to achieve a bottom-up component.As a result, it would perhaps be beneficial for additional information to be provided, particularly in terms of the practical experiences of the labs gained through the projects.For instance, how many stakeholders were represented, how many labs were held, for how long did they last and were there any challenges encountered through these in terms of participation or similar?Reflections along these lines -even if only brief at this time -may further help others in thinking about, designing and perhaps implementing such an approach that holds important potential to support adaptation monitoring plans.
Regarding the ARSINOE living labs, it is mentioned that stakeholders were identified through the quadruple helix 'based on [their] interest and influence' (first paragraph, Demo-site level monitoring section).It notes also that only the 'highest ranked' stakeholders were included.Maybe these could be elaborated upon through adding detail about what criteria was used to define a stakeholder as high-ranked, what qualifies as a relevant interest and how influence was understood and observed in practice.Living labs are nonetheless well-discussed as an approach later in the Letter, in the Stakeholder engagement section.
It may be useful to know more about the project contexts regarding what kinds of physical environments (natural and/or built) are being focused on, including the type of adaptation projects that are to be monitored.For instance, is the focus on coastal adaptation to erosion and/or flood risk in Europe, or a different adaptation challenge elsewhere?Whether the projects relate to hard and/or soft adaptation options and similar details may help enhance understandings of the contexts from which the Letter's important insights are drawn.
In the 'Data for monitoring' section it is noted how time lag can pose a challenge to adaptation outcome evaluation, but is there anything from the projects that could point towards possibly fruitful ways of addressing this?Maybe approaches complementary to (quasi-)experimental methods could be of value here?This would be interesting to hear slightly more about as part of the reflections and lesson-learning offered, including in relation to qualitative data.
Another keen insight concerns the importance of a clear use of terminology and shared understandings of key concepts in multi-stakeholder and multi-disciplinary settings.This is correctly underlined since the vocabulary can be used to emphasise different values and factors, reflecting also prior experiences and knowledge.Such differences may come, for instance, in understanding vulnerability in terms of economic loss or less tangible place attachments, or differences between qualitative and quantitative approaches to measurement and evaluation.Achieving a clear, collective set of definitions is therefore among the key suggestions presented by the Letter.
Indeed, the stated intention of this Letter is to present learning points from across the three projects and this appears to be well-done.The discussion, including the limitations and suggestions covered, seems both relevant and important to monitoring and impact evaluation questions in the climate adaptation arena.This Letter therefore represents a clear and meaningful contribution that can begin to help guide how future adaptation projects think about monitoring, especially regarding multi-stakeholder and qualitative-quantitative approaches to data and indicator development that are more site-specific and, perhaps, useful in practice.It will certainly be interesting to read more about the outcomes of the three projects as they continue to near completion.
Is the rationale for the Open Letter provided in sufficient detail?(Please consider whether existing challenges in the field are outlined clearly and whether the purpose of the letter is explained) Yes

Does the article adequately reference differing views and opinions?
Yes Are all factual statements correct, and are statements and arguments made adequately supported by citations?Yes Is the Open Letter written in accessible language?(Please consider whether all subjectspecific terms, concepts and abbreviations are explained) Partly Where applicable, are recommendations and next steps explained clearly for others to follow?(Please consider whether others in the research community would be able to implement guidelines or recommendations and/or constructively engage in the debate) Yes Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Environmental and Climate Governance, including Adaptation; Civil Society; Social Science Methods I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Reviewer Report 18
June 2024 https://doi.org/10.21956/openreseurope.18775.r41284© 2024 Tompkins E. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Emma Tompkins Southampton University, Southampton, UKThe paper reviews the lessons learned by 3 EU Horizon 2020 projects in trying to monitor adaptation.There is a little internal inconsistency of aim which needs some small tweaks to resolve.Is this paper about (i) monitoring the process and practice of adaptation, i.e. seeing what adaptation looks like in practice as it unfolds as the title infers, and as is stated in the introduction ("monitor and evaluate adaptation actions taken").Other examples of researchers doing this are:[3].Science and policy lessons learned from a decade of adaptation to the emergent risk of sargassum proliferation across the tropical Atlantic.[3] -shows the process of adaptation over a 10 year period), Or, is this paper about (ii) monitoring the "development towards climate resilience" i.e. how societies can be better prepared to live with climate change.Or, is it about (iii) identifying the challenges in monitoring adaptation -which is the focus of most of the rest of the paper.These are all worthy aims but they are different aims.Some clarification of text in the introduction (and e. content should include: what is the topic of interest, what is the research gap, what is the aim of the paper, what are the methods and materials, and what are the key findings. Open Letter provided in sufficient detail?(Please consider whether existing challenges in the field are outlined clearly and whether the purpose of the letter is explained) Partly Does the article adequately reference differing views and opinions?Partly Are all factual statements correct, and are statements and arguments made adequately supported by citations?Partly Is the Open Letter written in accessible language?(Please consider whether all subjectspecific terms, concepts and abbreviations are explained) Partly Where applicable, are recommendations and next steps explained clearly for others to follow?(Please consider whether others in the research community would be able to implement guidelines or recommendations and/or constructively engage in the debate) Partly Competing Interests: No competing interests were disclosed.Reviewer Expertise: climate change adaptation I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.Reviewer Report 17 June 2024 https://doi.org/10.21956/openreseurope.18775.r40827© 2024 Reckien D. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

©
2024 Garland J.This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Joshua Garland LUCSUS (Lund University Centre for Sustainability Studies), Lund University, Lund, Sweden This Open Letter, entitled 'Deriving lessons learned from monitoring adaptation activities in projects under the EU mission on adaptation', focuses on climate adaptation monitoring as an important challenge in evaluating the outcomes of adaptation efforts.It aims to inform approaches to this based on learning from three ongoing Horizon 2020-funded projects: ARSINOE; TransformAr; IMPETUS.The insights provided emerged through expert workshops concerning monitoring experiences and centre on four central areas: local relevancy; data; interdisciplinarity; stakeholder engagement.
Climate change affects different biogeographical regions, systems and sectors in diverse ways, whereas the projects are active in various regions and demo-on how to tailor them or how to work with data gaps.Thus, it is helpful to start with a smaller number of indicators and build up the set as experience grows.
sites.Considering this, it is challenging to provide a comprehensive and exhaustive list of indicators with defined targets that measure resilience.To address this challenge, a number of approaches have proven useful: -Create synergies with existing global frameworks: Existing monitoring systems at the global level (e.g.Sendai Framework (UNDRR, 2015), SDGs (United Nations, 2015)), European level (adaptation reporting system for the EU governance of the energy union and climate action (European Union, 2018)) regional, and local level (city networks e.g.C40 cities (C40 Cities, 2020)) provide a robust foundation for planning indicator-based monitoring of climate resilience.-Specify indicators: Using indicators of global frameworks offer comparability across diverse contexts, e.g.SDG indicators, promoting scalability and replicability.However, such a usage of broad, global indicators might fail to account for local/projectspecific/regional contexts.Site-specific/ biogeographical indicators provide more accurate insights.Projects operating across various contexts and demonstration sites should prioritize a deep understanding of local requirements.This involves engaging experts and stakeholders in an iterative process to select indicators that are relevant and meaningful to each specific location.While customizing monitoring approaches to individual sites or biogeographic regions is advisable, excessive diversity in indicator subsets presents a challenge.This diversity can hinder comparability and impede a comprehensive assessment of project progress and resilience objectives across demo-sites.-Prioritize indicators: Providing a comprehensive and exhaustive list of indicators for monitoring adaptation and vulnerability is extremely challenging due to the different sectors and systems that are affected by climate change in different biogeographical regions.Also, a framework is difficult to apply and adapt when it consists of too many indicators and does not provide guidance on selecting the most relevant ones or

Quadruple Helix Collaboration in practice.
R, Hagan KC, Gerhardus A:

EU) 2018/1999 of the European parliament and of the council of 11 December 2018 on the governance of the energy union and
climate action.2018.Reference Source Food and Agriculture Organization of the United Nations: Water-food-energy nexus.n.d.; Retrieved 07 Feb 2024.Reference Source Fotopoulou E, Mandilara I, Zafeiropoulos A, et al.: SustainGraph:

a knowledge graph for tracking the progress and the interlinking among the sustainable development goals' targets. Front
Environ Sci.2022; 10: 1-20.Publisher Full TextHammill A, Dekens J,Leiter T, et al.: Repository

of adaptation indicators. Real case examples from national monitoring and evaluation systems. GIZ
Sanchez Martinez G, Christiansen L, Naswa P: Adaptation metrics: perspectives on measuring, aggregating and comparing adaptation results.Copenhagen, Denmark: UNEO DTU Partnership, 2018.Reference Source Schuurman D, Stefaniga S, Piana V, et al.: Proceedings of the OpenLivingLab Days conference 2023.Brussels: European Network of Living Labs, 2023.Reference Source Setzer J, de Murieta ES, Galarraga I, et al.: Transnationalization

of climate adaptation by regional governments and the RegionsAdapt initiative.
1.The authors correctly highlight that monitoring and evaluation (M&E) serve multiple purposes in adaptation projects.In a cyclical policy and agenda-setting process, insights from M&E provide crucial knowledge for shaping and refining adaptation policies and measures.Understanding what works, under what conditions, and why, empowers regions to enhance their adaptation policies.M & E also builds accountability, especially towards the most vulnerable.It ensures that actions are equitable and support vulnerable communities without unintentionally exacerbating inequalities(Lager et al. 2023).Additionally, effective M&E can enhance our understanding of evolving climate risks.By bridging knowledge gaps and illuminating critical challenges and opportunities, M&E lays a robust groundwork for effective adaptation planning (Leitner et al.

Is the rationale for the Open Letter provided in sufficient detail? (Please consider whether existing challenges in the field are outlined clearly and whether the purpose of the letter is explained) Partly Does the article adequately reference differing views and opinions? Partly Are all factual statements correct, and are statements and arguments made adequately supported by citations? Yes Is the Open Letter written in accessible language? (Please consider whether all subject- specific terms, concepts and abbreviations are explained) Yes Where applicable, are recommendations and next steps explained clearly for others to follow? (Please consider whether others in the research community would be able to implement guidelines or recommendations and/or constructively engage in the debate) Partly Competing Interests:
.." the readers asked him/herself why.Please provide your reasoning, e.g. by providing examples.How did you get to this conclusion?No competing interests were disclosed.

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Reviewer Report 14 June 2024 https://doi.org/10.21956/openreseurope.18775.r41280